Linear transformations

• refer to constant changes across values that do not alter the relationship between these values
  • adding, subtracting, or multiplying by a constant value
• let’s look at some common ways of linearly transforming our data, and the reasons behind doing so

Centering

• Centering is typically applied to predictor variables
  • subtracting the mean of a variable from each value
  • results in each centered value representing the original value’s deviance from the mean (i.e., a mean-deviation score)
• What would a centered value of \(0\) represent in terms of the original values?

Centred predictor

• re-fit our model with and without centred predictor

# run our model with the original predictor
fit_rt_freq <- 
  lm(rt ~ freq, data = df_freq)

# run our model with the centered predictor
fit_rt_freq_c <- 
  lm(rt ~ freq_c, data = df_freq)

Standardizing (z-scoring)

• standardize continuous predictors
  • dividing centered values by the standard deviation of the sample
  • when to do this? if you have multiple continuous predictors (which we don’t at present)

Non-linear transformations

• the meat and potates of dealing with continuous variables (depending on your subfield)
• in linguistic research, and especially experimental research, we often deal with continuous variables truncated/bound at 0
  • Reaction times, reading times and formant frequencies: there is (typically) no such thing as a negative reading time or fundamental frequency
• these types of data are almost never normally distributed, typically having a ‘positive skew’ (long tail to the right)
  • this has implications for the normality of residuals fit to a straight line
  • these very large, exceptional values will have a stronger influence on the line of best fit, leading to the coefficient estimates that are “suboptimal for the majority of data points” [@Baayen (2008); p. 92]
• How do we deal with this nonnormality?
  • We use non-linear transformations, the most common of which is the log-transformation

Log-transformation

• luckily, we can easily log-transform continuous values by passing them through the log() function
  • this uses the natural logarithm, which finds the power to which Euler’s number (\(e\) = 2.718281828459) is raised to equal \(x\) (don’t worry about the math)
• importantly, log-transforming a continuous variable makes numbers smaller, with a larger shrinkage for larger numbers

Log word frequency

df_freq$freq

 [1] 55522 40629 14895  3992  3850   409   241   238    66    32     4     4

Log reaction times

df_freq |> 
  mutate(log_rt = log(rt)) |> 
  arrange(rt) |> 
  select(rt, log_rt) 

# A tibble: 12 × 2
      rt log_rt
   <dbl>  <dbl>
 1  507.   6.23
 2  520.   6.25
 3  587.   6.38
 4  611.   6.42
 5  622.   6.43
 6  637.   6.46
 7  705    6.56
 8  725.   6.59
 9  764.   6.64
10  794.   6.68
11  810.   6.70
12  878.   6.78

Model with log-transformed variables

df_freq <-
  df_freq |> 
    mutate(rt_log = log(rt),
           freq_log = log(freq),
           freq_log_c = freq_log - mean(freq_log))

fit_log <- lm(rt_log ~ freq_log_c, data = df_freq)

# or, log-transform directly in the model syntax
fit_log <- lm(log(rt) ~ freq_log_c, data = df_freq)

tidy(fit_rt_freq_c)

# A tibble: 2 × 5
  term         estimate std.error statistic  p.value
  <chr>           <dbl>     <dbl>     <dbl>    <dbl>
1 (Intercept) 680.       30.2         22.5  6.71e-10
2 freq_c       -0.00338   0.00170     -1.99 7.46e- 2

tidy(fit_log)

# A tibble: 2 × 5
  term        estimate std.error statistic  p.value
  <chr>          <dbl>     <dbl>     <dbl>    <dbl>
1 (Intercept)   6.51     0.0277     235.   4.72e-20
2 freq_log_c   -0.0453   0.00871     -5.20 4.03e- 4

• what has changed?

Extracting predictions

• the inverse of the log is the exponential

log(2)

[1] 0.6931472

exp(0.6931472)

[1] 2

• we can plus our equation of a line into the exp() function to extract predictions
  • what’s our predicted reaction time for the word door?

\[ \begin{align} y_i & = b_0 + b_1x_i y_i & = b_0 + b_1*freq(door) \end{align} \]

b0 <- coef(fit_log)['(Intercept)']
b1 <- coef(fit_log)['freq_log_c']
freq_door <-
  df_freq |> filter(word == "door") |> select(freq_log_c)

b0 + b1*freq_door

  freq_log_c
1   6.356246

exp(b0 + b1*freq_door)

  freq_log_c
1   576.0795

predict(fit_log)

       1        2        3        4        5        6        7        8 
6.296675 6.310814 6.356246 6.415861 6.417500 6.519012 6.542958 6.543525 
       9       10       11       12 
6.601596 6.634371 6.728517 6.728517 

augment()

• remember the augment() function appends model output to the data frame

df_freq <-
augment(fit_log, data = df_freq) |> 
  arrange(freq) |>  
  mutate(exp_fit = exp(.fitted)) |> 
  relocate(exp_fit, .after = rt)
df_freq |> 
  select(word, rt_log, .fitted, rt, exp_fit) |> 
  head()

# A tibble: 6 × 5
  word     rt_log .fitted    rt exp_fit
  <chr>     <dbl>   <dbl> <dbl>   <dbl>
1 nihilism   6.64    6.73  764.    836.
2 puffball   6.78    6.73  878.    836.
3 gnome      6.70    6.63  810.    761.
4 mocha      6.59    6.60  725.    736.
5 bloke      6.68    6.54  794.    695.
6 kitten     6.42    6.54  611.    694.

Log for positive values

• notice that we logged freq before centering it
  • this is because centering makes half the values negative (because half are below the mean)
  • it’s not possible to log-transform zero or negative numbers (because Eulen’s \(e\) cannot be risen to the power of 0 or a negative number!)
  • so, always log before centering predictors!

Task

Assessing assumptions

1. Re-run the models fit_rt_freq, fit_rt_freq_c, and fit_log
2. Produce diagnostic plots for each of them (histograms, Q-Q plots, residual plots)
3. Interpret the plots

Model comparison

1. Use the glance() function to inspect the \(R^2\), AIC, and BIC of each model.
2. Which is the best fit? Why?